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vv  e  introduce  a  technique  to  aid  in  interpreting 
infrared  satellite  images  of  the  North  Atlantic 
Ocean  Gulf  Stream  region.  Present  interpretive 
methods  are  largely  manual,  require  significant 
effort,  and  are  highly  dependent  on  the  interpreters 
skill.  Our  quasiautomated  technique  is  based  on 
mathematical  morphology,  specifically  the  image 
transformations  of  opening  and  closing,  which  are 
defined  in  terms  of  erosion  and  dilation.  The  imple¬ 
mentation  performs  successive  openings  and  clos¬ 
ings  at  increasing  thresholds  until  a  stable  division 
into  objects  and  background  is  found.  This  method 
finds  the  North  Wall  of  the  Gulf  Stream  in  approxi¬ 
mately  the  same  place  as  human  analysts  and 
another  automated  procedure,  and  does  less 
smoothing  of  small  irregularities  than  the  other 
two  methods.  The  North  Wall  is  continuous  and 
sharp  except  where  obscured  by  clouds.  Perfor¬ 
mance  in  locating  warm-core  eddies  is  also  compa¬ 
rable  to  the  other  methods.  However,  the  present 
procedure  does  not  find  cold-core  rings  well.  We 
are  presently  investigating  ways  to  reduce  the 
effects  of  clouds  and  delete  the  unwanted  water 
areas  found  by  the  method.  We  expect  to  be  able 
to  improve  the  cold-core  eddy  performance. 
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INTRODUCTION 

Infrared  satellite  images  of  the  ocean  provide 
surface  temperature  measurements  which  can  he 
list'd  either  to  supplement  local  measurements  at 
various  depths  obtained  by  conventional  oceano¬ 
graphic  Lrhniqucs  or  to  provide  information 
about  areas  of  the  ocean  where  data  from  conven¬ 
tional  techniques  is  sparse.  Figure  1  shows  a 
sample  infrared  image  of  the  Gulf  Stream  area  of 
the  Atlantic  Ocean.  Bright  areas  in  this  image 
correspond  to  low  temperatures,  either  of  the 
ocean  .surface  or  of  clouds. 

The  Gulf  Stream  transports  tropical  water  and 
heat  over  large  distances,  and  drives  main  physi¬ 
cal  processes  in  the  Atlantic.  Areas  of  closed  circu¬ 
lation  (vortices)  near  the  Gulf  Stream,  called  rings 
or  eddies,  form  part  of  the  Gulf  Stream  system. 
The  Gulf  Stream  and  its  associated  eddies  are 
examples  of  mesoscale  (50-300  km  in  size)  ocean 
features. 

Presently,  interpretation  of  satellite  ocean  im¬ 
ages  is  performed  largely  by  humans.  Conse¬ 
quently,  interpretation  is  a  lengthy  and  laborious 
process,  as  well  as  subjective  in  nature  and  highly 
dependent  on  the  interpreter's  skill.  To  address 
this  problem,  an  automated  sy  stem  is  being  devel¬ 
oper!  to  interpret  mesoscale  features  for  these 
images.  The  system,  or  components  of  it.  are 
described  by  Lybanon  et  al.  (1986),  Krisbnakumar 
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Figure  1.  An  original  image  ol  the  (hill  Stream  area  til  the 
Atlantic  Ocean  from  NOAA-9.  The  image  is  a  warmest  - 
pixel  composite  of  images  acquired  7-8  March  1987 
Bright  areas  in  the  image  correspond  to  low  temperatures. 
The  image  is  displayed  using  histogram  equalization  to  en¬ 
hance  the  contrast. 


et  al.  (1990),  and  Lvbanon  et  al.  (1990).  This 
system  divides  the  problem  into  four  parts:  seg¬ 
mentation,  feature  labeling,  feature  synthesis,  and 
mesoscale  extrapolation. 

The  segmentation  portion  of  the  system  pres¬ 
ently  consists  of  edge  detection  using  the  grey- 
level  cooccurrence  matrix,  as  described  by  Holyer 
and  Feckinpaugh  (1989;  Peckinpaugh  and  Holyer, 
1991).  Gayula  et  al.  (1991)  compare  that  method 
with  another.  Cambridge  (1991)  and  Lebanon  et 
al.  (1990)  discuss  incorporation  of  region  detec¬ 
tion  into  this  stage  (see  also  Haddon  and  Bovee, 
1990).  The  feature  labeling  process  uses  a  relax¬ 
ation  labeling  technique  due  to  Krishnakumar  et 
al.  (1990),  and  Krishnakumar  and  Iyengar  (1991); 
the  use  ol  genetic  algorithms  is  also  being  investi¬ 
gated,  as  described  by  Buckles  and  Petry  (1991). 
Feature  synthesis  combines  discontinuous  labeled 
edges  into  continuous  features,  such  as  a  normal 
mode  reconstruction  of  the  Gulf  Stream’s  North 
Wall,  presented  by  Molinelli  and  Flanigan  (1987). 
JMolinelli  et  al.  (1991)  also  discuss  the  use  of 
neural  networks  to  synthesize  North  Walls.]  The 
mesoscale  extrapolation  module  uses  an  expert 
system  to  predict  kinematically  the  motion  of 


eddies,  since  numerical  models  based  on  phy  sical 
principles  cannot  yet  predict  eddy  motion  well 
The  latter  problem  is  discussed  by  Lybanon  and 
Thompson  (1991),  and  Lybanon  et  a!.  (1990). 

The  present  yvork  is  relevant  to  the  segmenta¬ 
tion  and  synthesis  portions  of  the  system:  This 
article  discusses  the  use  of  mathematical  morphol¬ 
ogy  to  find  eddies  and  the  North  Wall  of  the 
Gulf  Stream.  Previous  work  in  this  area  had  used 
primarily  statistical  pattern  recognition  tech¬ 
niques  and  decision  theory  [see  Harulii  k  (1982); 
later  work  appears  in  Gersou  et  al,  (1982).  Coulter 
(1983),  Janovvitz  (1985),  and  Niehol  (1957).) 
Szezechowski  (1991)  discusses  using  the  Mari - 
Hildreth  operator  to  detect  eddies. 

DATA  AND  PROCESSING 

Infrared  images  of  the  Gull  Stream  area  ol  the 
Atlantic  Ocean  are  obtained  from  Ch  umei  f 
(10.3-1 1.3  /mi)  of  the  Advanced  Very  High  Reso¬ 
lution  Radiometer  (AVHRR)  aboard  the  NOAA-9 
and  NOAA-11  satellites.  This  instrument  pro¬ 
duces  2048  10-hit  samples  per  scan  line  in  each 
channel.  The  resulting  images  are  processed  to 
512x512  pixels  with  8  hits  of  intensity.  They  art- 
51  2  x  512-pixel  sections  with  a  pixel  size  of  2.5 
kin.  Tht-  8-bit  intensity  range  covers  the  tempera¬ 
ture  range  of  interest  with  the  instrument's  lull 
thermal  resolution.  Tht'  images  used  in  the  study 
are  warmest-pixel  composites  of  two  or  more  con¬ 
secutive  images,  to  reduce  cloud  cover. 

The  algorithm  used  to  find  eddies  anti  the 
North  Wall  of  the  Gulf  Stream  is  based  on  the 
image  transformations  of  opening  and  closing  de¬ 
veloped  by  Serra  (1982)  in  connection  with  tex¬ 
ture  analysis  of  digital  images.  Useful  rev  iews  and 
discussions  maybe  found  in  Haralick  et  al.  (1987) 
and  Wilson  (1989).  A  complete  description  of  this 
algorithm  and  its  implementation  for  astronomic  al 
images  is  given  by  Lea  and  Kellar  (1989).  Lea 
(1991)  discusses  adaptations  of  the  algorithm  re¬ 
quired  for  its  use  with  infrared  satellite  images  of 
oceans;  these  adaptations  are  summarized  below. 

The  operations  of  opening  and  closing  are 
defined  in  terms  ol  the  familiar  erosion  and  dila¬ 
tion  operations.  Tile  operations  can  best  be  under¬ 
stood  by  considering  a  single  row  of  an  image 
(Fig.  2).  The  structure  element  (a  3-pixel-wide 
rectangle,  in  this  case)  slides  along  the  line,  with 
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Figure  2.  Thr  erosion  and  dilation  operations,  a)  The  structure  element  is  the  rectangle  3  pixels  wide  by  -t  intensity  units 
high,  b)  One  line  of  an  image  16  pixels  wide  is  displayed.  The  structure  element  slides  along  this  line,  as  discussed  in  the 
text,  c)  The  resulting  eroded  image  (see  text),  d)  The  resulting  dilated  image  (see  text),  e)  The  resulting  opened  and 
closed  image  (see  text). 


its  center  corresponding  successively  to  pixels  3x3  may  he  used,  but  the  3x3  size  gives  the 

2-15  of  the  image.  For  erosion,  when  the  entire  smoothest  edges  on  the  images  used, 

rectangle  fits  under  or  at  the  image  intensity  line.  An  opening  is  an  erosion  followed  by  a  dilation 

the  pixel  in  the  transformed  image  corresponding  of  the  eroded  image.  A  closing  is  a  dilation  fol- 

to  the  center  pixel  of  the  rectangle  is  given  the  lowed  by  an  erosion  of  the  dilated  image.  The 

minimum  intensity  of  the  pixel  and  its  two  neigh-  implementation  opens  the  image  and  then  closes 

hors  in  the  original  image;  otherwise,  the  pixel  is  the  opened  image  at  a  particular  threshold  (box 

set  to  zero  intensity.  (A  structure  element  having  height).  The  output  of  this  step  is  the  input  to 

odd  width  n  would  he  compared  to  n  -  1  neigh-  the  next  step,  opening  and  closing  at  a  higher 

hors.)  For  dilation,  if  any  pixel  of  the  rectangle  threshold.  The  process  is  repeated  until  a  stable 

fits  under  or  at  the  image  intensity  line,  the  corre-  division  into  objects  and  background  has  been 

spending  center  pixel  is  given  the  maximum  in-  found.  The  division  is  considered  stable  when 

tensity  of  the  pixel  and  its  two  neighbors  in  the  no  pixels  change  classification  (from  object  to 

original  image;  otherwise,  the  pixel  is  set  to  zero  background  or  the  reverse)  between  the  begin- 

intensity.  In  three  dimensions,  the  rectangle  be-  ning  of  an  opening  and  closing  step  and  its  end. 

comes  a  box,  and  there  are  eight  neighbors  to  The  implementation  will  not  find  eddies  di- 

be  considered  in  determining  the  minimum  or  rectly  in  the  ocean  images  for  three  reasons:  First, 

maximum.  Structure  elements  of  size  larger  than  the  implementation  expects  to  find  bright  objects 
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against  a  dark  background;  second,  it  expects  to 
find  objects  which  are  uniform  or  increase  in 
intensity  toward  their  centers;  and  third,  it  ex¬ 
pects  to  find  (or  reject)  tin-  brightest  objects  in 
the  image.  (In  other  words,  the  implementation 
was  originally  designed  to  find  stars  in  astronomi¬ 
cal  images.)  Although  it  might  be  preferable  to 
adjust  the  algorithm  to  find  eddies  directly  ,  doing 
so  would  require  knowing  the  approximate  inten¬ 
sity  range  of  the  eddies  before  processing.  Acquir¬ 
ing  this  information,  however,  requires  finding 
the  eddies  themselves.  Fortunately,  prepro¬ 
cessing  the  NOAA  images  makes  use  of  tin*  ex¬ 
isting  implementation  possible. 

The  initial  preprocessing  step  is  to  invert  the 
image,  so  that  regions  of  high  intensity  correspond 
to  hot  objects.  Since  the  opening  and  closing 
operations  are  used  to  remove  both  very  hot  and 
cold  objects,  it  might  seem  that  this  step  is  unnec¬ 
essary;  in  practice,  however,  the  implementation 
finds  warm-core  eddies  more  reliably  if  the  images 
are  inverted.  This  increased  reliability  is  ascribed 
to  the  fact  that  warm-core  eddies  look  more  like 
stars  in  inverted  images. 

After  image  inversion,  the  opening  and  closing 
operations  are  used  to  find  and  remove  very  hot 
objects  (usually  land  and  the  Gulf  Stream).  Opera¬ 
tion  of  this  stage  of  the  processing  is  speeded 
up  by  starting  the  iteration  at  a  relatively  large 
intensity  threshold.  An  estimate  of  its  value  can 
be  determined  from  the  unprocessed  image: 

t-  120. 7(m  /255)  +  0.30a  -  2»,  (1) 

where  t  is  the  threshold,  m  is  the  largest  intensity 
present  in  the  image,  a  is  the  average  image 
intensity,  and  n  is  the  average  of  the  square  roots 
of  the  intensities.  [The  coefficients  of  m  and  a  in 
Eq.  (1)  may  vary  with  season;  the  equation  is  a 
multivariate  least-squares  fit  to  the  set  of  test 
images,  which  were  mostly  obtained  during  April 
and  May.]  For  best  results,  the  criterion  for  sepa¬ 
rating  objects  from  background  needs  to  be  re¬ 
laxed.  Exactly  two  iterations  are  performed;  if 
0.2%  or  more,  but  fewer  than  1.0%  of  the  pixels 
are  still  changing  classification,  a  new  threshold 
is  estimated  from  the  visual  results,  and  two  itera¬ 
tions  are  done  starting  at  that  value.  This  process 
is  repeated  until  the  results  are  satisfactory.  Some 
pixels  remain  ambiguously  classified  as  a  conse¬ 
quence  of  using  only  two  iterations;  however,  in 
the  images  studied,  99.3-99.8%  of  the  pixels  are 


unambiguously  classified  as  a  result  of  this  proce¬ 
dure. 

In  the  output  image,  pixels  in  objects  found 
are  set  to  0  intensity,  and  pixels  in  the  background 
have  their  intensities  set  to  the  v  alue  in  the  origi¬ 
nal  image.  Ambiguous  pixels  are  assigned  to  what¬ 
ever  category  they  happen  to  occupy  at  the  time 
the  iteration  is  stopped.  The  behav  ior  of  the  algo¬ 
rithm  at  the  edges  of  objects  causes  a  1 -pixel-wide 
border  to  be  discarded  from  this  output  image. 

The  output  image  from  the  pievious  pro¬ 
cessing  step  is  used  as  the  input  image  for  tin* 
final  processing  step.  In  this  step,  the  opening 
and  closing  operations  are  used  again,  with  both 
a  smallei  increment  and  a  smaller  initial  threshold 
(since  the  brightest  objects  have  been  remov  ed), 
to  separate  remaining  hot  objects  from  the  colder 
background  and  remove*  the*  background.  Opera¬ 
tion  of  this  stage  of  the  processing  is  speeded  up 
both  by  keeping  only  sizable  objects  (e  g..  25  x  25) 
and  by  starting  the  iteration  at  a  nonzero  intensity 
threshold.  The  value  of  this  threshold  can  be* 
estimated  from  the  output  images  of  the  first  step: 

t  =  132.6(/n  /  255)  -  0.36«  -  2n.  (2) 

[Equation  (2),  like  Eq.  (1).  is  a  multivariate  least- 
squares  fit  to  the  set  of  output  images  resulting 
from  performing  the  first  processing  step  on  the 
set  of  test  images.  Consequently  ,  tin*  coefficients 
of  m  and  a  may  vary  with  season  here  also.] 
The  relaxed  criterion  for  separating  objects  from 
background  is  used  here  as  well. 

In  the  output  image  from  this  step,  pixels  in 
objects  are  assigned  the  intensity  of  the  original 
image,  and  pixels  in  the  background  are  assigned 
intensity  values  of  0.  Ambiguous  pixels  again  are 
assigned  to  whatever  category  they  occupy  at  the 
time  the  iteration  stops.  A  1  -pixel-wide  border  is 
also  removed  from  this  output  image. 

RESULTS  AND  DISCUSSION 

Figures  3-6  show  the  results  of  using  the  imple¬ 
mentation  on  various  images.  In  each  ease,  image 
a  is  the  original  (inverted)  image,  image  b  is  image 
a  with  bright  objects  remov  ed,  showing  the  North 
Wall  of  the  Gulf  Stream,  and  image  e  is  image  b 
with  cold  objects  removed.  Image  d  (if  present) 
shows,  for  comparison,  the  results  of  a  human 
analy  sis  and  another  automated  analysis.  The  lat- 
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clouds  aetnallv  obscure  it  The  prncedme  does 
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i'inun  I  Nil  iniat'cs  arc  display'd  nsiim  liislnip.ini  ei|n.ili/.ili<m  lit  enhance  lire  contrast:  a)  an  inverted  warniest-pixel 
i  nmposile  nl  inianes  at  <  |i  iir«*il  I  is  I  In-  NONA- 1  I  satellite.  10  1 2  April  1 9  VI,  l>)  I  In'  \  ml  it  Wall  nl  I  lie  ( hill  Stream  Ini  mil 
In  (lie  procedure  alter  the  first  opening  and  closiim  nperalinii:  <1  eddies  liiinid  li\  the  prneednre  alter  the  seeend  opening 
and  closinu  nperalinii.  d)  the  \nrlh  Wall  and  warm-core  eddies  himid  In  a  human  anahst  perl)  and  In  another  automated 
procedure  lUieenl 


toil  interpolate  where  rdijc  sections  are  obscured. 
ImHUIc  le  shows  eddies  found.  The  warm-core 
eddies  are  found  well,  hut  a  luri'e  aiiiounl  of 
unwanted  cloud  cmer  is  found  In  the  procedure 
as  well,  himire  Id  shows  (lie  \orlli  W  all  and  one 
warm cote  edd\  found  In  a  human  auahsl  and 
in  the  other  automated  technii|iie. 


I’iuure  na  shows  an  inserted  composite  imaiie 
from  2 1  -2d  April  I  DSP.  This  inutile  is  lairlx  t\  pieal 
in  its  amount  of  cloud  eo\er.  I'inure  db  show  s  the 
North  W  all  found.  A.uaiu,  it  is  clear  except  where 
clouds  aetualK  eoxer  the  edye.  The  North  W  all 
found  agrees  more  eloseb  with  that  indicated  In 
the  human  anabsl  in  Kinnre  dd  than  does  the 
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l  iftin’  t>  Kddics  found  using  an  uniiiserted  version  of  pig 
lire  oa  The  threshold  used  ssas  specialized  to  the  intake. 
The  image  is  displaced  using  histogram  ei|iiali/atiou  to  en¬ 
hance  the  contrast. 


verted)  images  to  find  eold-eore  eddies  was  made. 
Figure  0  shows  the  results  for  the  image  ol  21-23 
April.  The  eold-eore  eddies  are  lound  by  the 
procedure  in  this  test:  however,  since  they  match 
the  water  around  them  closely  in  temperature, 
large  amounts  of  this  water  are  lound  in  addition. 
Work  is  in  progress  on  developing  a  general 
method  to  find  both  warm-core  and  eold-eore 
eddies. 


SUMMARY 

The  method  finds  the  North  Wall  of  the  Ciulf 
Stream  in  approximately  the  same  place  as  human 
analysts  and  another  automated  procedure,  gener¬ 
ally  better  than  the  latter.  Warm-core  eddies  are 
found  as  well  by  this  method  as  by  the  otlu  r 
methods.  A  major  advantage  of  this  method  is 
that  the  edges  found  are  continuous  unless  inter¬ 
rupted  by  cloud  cover.  The  edges  are  also  sharp: 
Border  pixels  are  all  of  zero  intensity.  However, 
when  eddies  are  associated  with  trails  of  water  of 
approximately  the  same  temperature,  the  method 
does  not  separate  eddies  from  trails.  It  also  finds 
some  unwanted  land  areas  and  cloud  cover.  It 
should  be  noted,  however,  that  no  shape  informa¬ 


tion  lias  been  included  to  discriminate  between 
eddies  and  trails,  nor  between  eddies  and  un¬ 
wanted  objects,  also,  no  "masking-'  of  land  or 
clouds  was  used 

f  he  performance  of  the  method  on  cold-core 
eddies  is  not  good,  primarily  because  cold-core 
eddies  do  not  resemble  stars  as  closek  as  do 
warm-core  eddies.  We  can  improve  the  perfor¬ 
mance  on  cold-core  eddies  by  using  techniques 
which  make  them  look  more  like  stars 

We  conclude  that  this  method  show  s  promise 
of  being  useful  in  conjunction  ss  it h  other  portions 
of  the  automated  system,  which  complement  its 
weaknesses  niceh. 

/7ns  uork  u«s  purl  mill/  sjtotisie  ed  hit  tin  0(fui  <>/  \Wi  ul 
I'ii  Inn ilt /gu  timin'  I’h.  ttl>t)24  i)V  COR  /.ee  Hounds.  Program 
Manager.  The  authors  are  happi/  to  thank  S  /V<  kinpaugh  foi 
prot  ill  ini’  iinagem  ’Old  ihr  results  of  other  analuu  v  H  l -rant 
lor  itnaue  prut  i  ssing  support,  uml  J  Hint  kins  lor  useful  snggi  s 
lions  uhout  the  text  lie  also  thunk  an  ammifmotis  referee  for 
suggestions  uhieh  inuteriiilh/  unprut  rd  the  presentation  m 
this  paper  this  ilnnonrnt  is  approteil  for  publii  release . 
distribution  is  unlimited  This  is  Mil.  Contribution  l\  >21 
0472)2 
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